Nonnegative approximations of nonnegative tensors

We study the decomposition of a nonnegative tensor into a minimal sum of outer product of nonnegative vectors and the associated parsimonious näıve Bayes probabilistic model. We show that the corresponding approximation problem, which is central to nonnegative parafac, will always have optimal solutions. The result holds for any choice of norms and, under a mild assumption, even Brègman divergences. Journal of Chemometrics, vol.23, pp.432–441, Aug. 2009 1. Dedication This article is dedicated to the memory of our late colleague Richard Allan Harshman. It is loosely organized around two of Harshman’s best known works — parafac [19] and lsi [13], and answers two questions that he posed. We target this article to a technometrics readership. In Section 4, we discussed a few aspects of nonnegative tensor factorization and Hofmann’s plsi, a variant of the lsi model co-proposed by Harshman [13]. In Section 5, we answered a question of Harshman on why the apparently unrelated construction of Bini, Capovani, Lotti, and Romani in [1] should be regarded as the first example of what he called ‘parafac degeneracy’ [27]. Finally in Section 6, we showed that such parafac degeneracy will not happen for nonnegative approximations of nonnegative tensors, answering another question of his. 2. Introduction The decomposition of a tensor into a minimal sum of outer products of vectors was first studied by Hitchcock [21, 22] in 1927. The topic has a long and illustrious history in algebraic computational complexity theory (cf. [7] and the nearly 600 references in its bibliography) dating back to Strassen’s celebrated result [36]. It has also recently found renewed interests, coming most notably from algebraic statistics and quantum computing. However the study of the corresponding approximation problem, i.e. the approximation of a tensor by a sum of outer products of vectors, probably first surfaced as data analytic models in psychometrics in the work of Harshman [19], who called his model parafac (for Parallel Factor Analysis), and the work of Carrol and Chang [8], who called their model candecomp (for Canonical Decomposition). The candecomp/parafacmodel, sometimes abbreviated as cp model, essentially asks for a solution to the following problem: given a tensor A ∈ Rd1×···×dk , find an optimal rank-r approximation to A, (1) Xr ∈ argminrank(X)≤r‖A−X‖,